Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dm: openapi support multi TLS security config for downstream db and cluster #11844

Merged
merged 67 commits into from
Dec 31, 2024
Merged

dm: openapi support multi TLS security config for downstream db and cluster #11844

merged 67 commits into from
Dec 31, 2024

Conversation

River2000i
Copy link
Contributor

@River2000i River2000i commented Dec 9, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #11831 #11945

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

@ti-chi-bot ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/dm Issues or PRs related to DM. contribution This PR is from a community contributor. labels Dec 9, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Dec 9, 2024

Hi @River2000i. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot bot added needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 9, 2024
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 10, 2024
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 12, 2024
@River2000i River2000i changed the title [WIP]dm: openapi support multi TLS security config for connect downstream db and cluster [WIP]dm: openapi support multi TLS security config for downstream db and cluster Dec 12, 2024
@lance6716
Copy link
Contributor

/ok-to-test

@ti-chi-bot ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Dec 12, 2024
River2000i
River2000i commented Dec 31, 2024
View reviewed changes
dm/config/task.go
@@ -301,6 +302,9 @@ type LoaderConfig struct {
RangeConcurrency int `yaml:"range-concurrency" toml:"range-concurrency" json:"range-concurrency"`
CompressKVPairs string `yaml:"compress-kv-pairs" toml:"compress-kv-pairs" json:"compress-kv-pairs"`
PDAddr string `yaml:"pd-addr" toml:"pd-addr" json:"pd-addr"`
// now only creating task by OpenAPI will use the `Security` field to connect PD.
// TODO: support setting `Security` by dmctl
Security *security.Security `yaml:"-" toml:"security" json:"security"`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not support yaml format since it will block few engine test, also we not support set the filed by yaml.

@River2000i
Copy link
Contributor Author

/retest

2 similar comments
@River2000i
Copy link
Contributor Author

/retest

@River2000i
Copy link
Contributor Author

/retest

GMHDBJD
GMHDBJD reviewed Dec 31, 2024
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

dm/checker/checker.go
@@ -550,16 +547,6 @@ func (c *Checker) Init(ctx context.Context) (err error) {
return nil
}

func lightningCheckGroupOnlyTableEmpty(checkingItems map[string]string) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove this check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix this issue #11945. Root cause is this check will set pd addr to noop:2379. In fact, when we run checker by RunCheckOnConfigs() https://github.com/pingcap/tiflow/blob/master/dm/checker/cmd.go#L113, it will initial all checker and remove few ignored checkers https://github.com/pingcap/tiflow/blob/master/dm/checker/cmd.go#L49. But the rest checker do not ignored, will use pdClient to get some info.

@River2000i
Copy link
Contributor Author

@GMHDBJD @D3Hunter PTAL~

GMHDBJD
GMHDBJD approved these changes Dec 31, 2024
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 31, 2024
D3Hunter
D3Hunter approved these changes Dec 31, 2024
View reviewed changes
dm/config/security/security.go Show resolved Hide resolved
dm/config/security_test.go Outdated Show resolved Hide resolved
dm/loader/lightning.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 31, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Dec 31, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-12-31 09:21:36.249072195 +0000 UTC m=+428631.605076732: ☑️ agreed by GMHDBJD.
  • 2024-12-31 09:39:09.399219152 +0000 UTC m=+429684.755223688: ☑️ agreed by D3Hunter.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Dec 31, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, GMHDBJD, yudongusa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Dec 31, 2024
@ti-chi-bot ti-chi-bot bot merged commit 91902aa into pingcap:master Dec 31, 2024
26 checks passed
@River2000i River2000i deleted the dmMultiSecurityConfig branch January 1, 2025 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yudongusa yudongusa yudongusa approved these changes

@GMHDBJD GMHDBJD GMHDBJD approved these changes

@D3Hunter D3Hunter D3Hunter approved these changes

Assignees
No one assigned
Labels
approved area/dm Issues or PRs related to DM. contribution This PR is from a community contributor. lgtm ok-to-test Indicates a PR is ready to be tested. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

dm support multi security config for connect TiDB cluster components and tidb-server
6 participants
@River2000i @lance6716 @Benjamin2037 @D3Hunter @GMHDBJD @yudongusa